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Why Systemic Reform? 

In 1990, the National Science Foundation 
(NSF) created the Statewide Systemic Initia- 
tive Program. The solicitation issued by the 
Directorate for Science and Engineering Edu- 
cation sought proposals "for projects intend- 
ed to broaden the impact, accelerate the pace, 
and increase the effectiveness of improve- 
ments in science, mathematics, and engineer- 
ing education in both K-12 and post-sec- 
ondary levels"(NSF, 1990, p. 1). 

Projects funded as Statewide Systemic Ini- 
tiatives (SSIs) were to align various parts of 
the system to produce comprehensive, coor- 
dinated, and sustained change. Curriculum 
learning goals; content, instructional materi- 
als, and practice; assessment; teacher recruit- 
ment and preparation; and professional 
development of teachers, administrators, and 
others — all were to come under the umbrel- 
la of systemic reform, as were other parts of 
the system that affected "ways of doing busi- 
ness": organizational structure and decision 
making, allocation of resources, articulation 
within the system, and accountability. SSIs 
were also to involve an array of stakeholders 
in reform efforts — scientists and mathemati- 
cians, business and community representa- 
tives, local school system decision makers, 
and leaders of parent and community-based 
organizations. This CERE Policy Brief looks 
at some of the lessons from the SSI experience 
gleaned from a Horizon Research, Inc. (HRI) 
study of NSF's Statewide Systemic Initiative 
Program i 

1 For a fuller discussion of the results, see Heck, Weiss, Boyd, 
Howard, & Supovitz (2003). 



The systemic approach was intended to 
address two perceived shortcomings of past 
reform efforts. First, some analysts argue that 
education reform targeting isolated compo- 
nents of a system, even when successful, 
have been short-lived, primarily because 
pressures within the system quickly prompt 
a return to the status quo (e.g., Berman & 
McLaughlin, 1978; Fullan, 2001; Smith & 
O'Day, 1991). Second, more sophisticated 
efforts toward education reform — ones that 
address multiple components of the 
system — have often been tied to the vision 
and leadership of a single individual or 
source of funding. With the departure of the 
critical individual or the loss of the particular 
source of funding, some researchers suggest 
that reform within an education system will 
not endure (e.g.. Wiles, 1993). 

The expectation was that SSIs would 
design and implement reform efforts that 
moved beyond a single component of the 
system and beyond the vision of a single 
leader. A clear and widely shared vision 
would target and link multiple aspects of the 
system in a coordinated fashion to support 
deep and lasting change. SSIs would also 
need to devise ways to scale up interven- 
tions, develop new leaders and stable sources 
of funding, and tend to the politics of under- 
taking large-scale education reform. Much of 
this work was new territory for those 
charged with managing the SSIs. How would 
the principal investigators (Pis) of SSIs and 
others build consensus, "system" synergy 
and capacity, leadership, and the prospects 
for longevity? What can others engaged in 



Consortium for 
Policy Research 
in Education 

University 
of Pennsylvania 

Harvard University 

Stanford University 

University 
of Michigan 

University of 
Wisconsin-Madison 






Policy Briefs 



large-scale reform efforts learn from these 
experiences? The HRl study explored the 
strategies used to accomplish those goals. 

Studying Strategic-ness 

During the period 1991-1998, NSF funded 
26 SSls in amounts up to $2 million per year 
for five years; four of these initiatives were 
terminated early, while eight received fund- 
ing for a second five-year phase. The HRl 
study of the SSls pursued the "thinking" 
behind these systemic efforts — how and 
why they were designed, implemented, and 
evaluated in the ways that they were. SSI 
leaders and documents served as the prima- 
ry data sources.2 

We designed the study specifically to 
explore the notion of "strategic leadership." 
Our framework for examining this concept 
was informed by the literature on leadership 
for systems change and sustainable develop- 
ment in an array of fields, including educa- 
tion, business, nonprofit organizations, agri- 
culture, ecology, and national development. 
Many parallels were evident in the funda- 
mental principles underlying how leadership 
for systems change is conceived across these 
fields, for example, in the areas of infrastruc- 
ture development, capacity building, and 
equity. The literature offered perspectives 
that were helpful in considering what aspects 
to investigate and how to interpret our data 
about the thinking behind the SSls (e.g., 
Dovers, 1990; Farrell & Hart, 1998; Gold- 
smith, 1996; Hickson, Butler, Cray, Mallory, & 
Wilson, 1989; Koteen, 1989; Niu, Lu, & Khan, 
1993). 

We paid particular attention to three crite- 
ria of strategic leadership: (a) leaders' under- 
standing of what the SSI had the capacity to 

2 This study was funded at the same time as several other 
research studies involving the SSls. To maximize learning, 
while minimizing burden on the projects, we tried to reduce 
overlap as much as possible consistent with our study goals. 
We reviewed documents (proposals, progress reports, and 
evaluation reports) from 21 of the 26 SSls, excluding the four 
that were discontinued midway because of insufficient 
progress and the one that declined to participate, and 
focused new data collection on 13 SSls, some in greater depth 
than others. Interviews were conducted with as few as five 
and as many as seven leaders in each of six SSls (Arkansas, 
Maine, Michigan, Nebraska, Puerto Rico, and Vermont), and 
with one to three leaders in each of seven SSls (Connecticut, 
Delaware, Georgia, Ohio, New Jersey, South Carolina, and 
Texas). 



do well and how it could be used to leverage 
change within the system, (b) leaders' under- 
standing of the context in which the SSI oper- 
ated, and (c) the "fit" between directions pur- 
sued by SSI leaders and the contextual condi- 
tions they were trying to change (Goldsmith, 
1996). Our interpretations relied especially on 
theorists' conceptions of strategic thinking 
toward broad and sustainable change, 
including the extent to which leaders provid- 
ed direction, prioritized the use of resources, 
anticipated and coped with uncertainty, and 
maintained an objective basis for quality con- 
trol and evaluation of progress (Goldsmith, 
1996; Koteen, 1989; Shrivastava, 1985). 

Earlier studies of the SSls had highlighted 
a critical duality in how leaders thought 
about the initiatives. For example, proceed- 
ings from a conference of SSI leaders noted 
that "Systemic reform is as much a political 
enterprise as it is a technical one" (Horizon 
Research, Inc., Inverness Research Associ- 
ates, & Westat, Inc., 1994, p. vi). Similarly, in 
the overall evaluation of the SSI program, 
Zucker and colleagues (1995) wrote "the dif- 
ficulty of systemic reform stems from the fact 
that it presents both technical and political 
challenges. Technically, it requires tackling 
the toughest problems in a complex system of 
education. . . . Politically, systemic reform 
requires garnering professional and public 
support for the change agenda" (p. 47). The 
political work of systemic reform also 
required leaders to understand, work within, 
work around, and ultimately influence key 
policies that provide guidance and incentives 
at multiple levels in the system. 

Making Strategic Choices 

The HRl study sought to understand 
which aspects of the system leaders viewed 
as critical to change and why, and to under- 
stand how they acted accordingly. In design- 
ing and implementing their initiatives, SSI 
leaders made many choices — about target 
audiences, interventions, and so on. Not sur- 
prisingly, approaches to design and imple- 
mentation reflected differences in the back- 
ground, experience, and underlying thinking 
of SSI leaders. The choices were many. In 
addition to identifying targets for change, SSI 
leaders had to consider how best to allocate 
resources and sequence activities for maxi- 
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mum benefit. Since many projects worked 
directly with districts, schools, and teachers, 
but none could do extensive work with all, 
SSI leaders had to make choices that they rea- 
soned would have the best payoff. SSls also 
had to balance short- and long-term objec- 
tives, especially in light of the goal of sus- 
taining both the benefits of the reform and 
the reform process itself. In our study, we 
looked at the ways in which SSI leaders 
addressed short-term objectives to provide 
clear benefits and high visibility, even as they 
simultaneously pursued long-term goals for 
deep, structural change in the education sys- 
tem. 

SSI leaders had other important trade-offs 
to consider. Striking the appropriate balance 
between centralized guidance and local deci- 
sion making posed challenges, particularly in 
light of the fact that the context and history of 
the state often exerted considerable influence 
in this area. The theoretical basis for systemic 
education reform (Clune, 1993; Smith & 
O'Day, 1991) provided some direction on this 
trade-off, but leaders also had to consider 
how much centralized guidance districts and 
schools expected or would tolerate, how 
much capacity for local decision making 
existed and whether it was likely to be 
aligned with the reform vision, and how 
quality control over interventions and policy 
implementation would occur at the local 
level. 

The HRl study also looked at the internal 
and external forces that influenced leaders' 
thinking. For example, what were the com- 
plementary or conflicting interests of the 
internal SSI leadership, and how did these 
influence the rationale for the SSI design? 
Externally, SSI leaders had to consider the 
interests and tolerance for reform of diverse 
constituencies, and ensure that proposed 
changes met the perceived needs of different 
groups and benefited the education system 
as a whole. Whose attention, interest, and 
support did the SSls attempt to cultivate? 
How did leaders take into account the accept- 
ability of the proposed reforms to the public 
and professional stakeholders? A rational or 
public choice model, derived from contem- 
porary political theories (Heywood, 1999), 
provided a useful interpretive lens for under- 



standing this aspect of the SSls (despite the 
well-founded criticisms of these models that 
many decisions made about public services 
are based on incomplete information and 
ignore long-term consequences). Finally, we 
examined the extent to which SSI leaders 
developed and implemented a plan not sim- 
ply to "scale up" a set of reform interventions 
to reach more teachers, classrooms, and 
schools, but also to "go to scale" so that the 
guidance, incentives, and culture of the sys- 
tem were transformed to support ongoing 
reform (Elmore, 1996). 

As we came to understand it, strategic 
leadership requires a deep understanding of 
the system one is trying to change. Strategic 
leaders know who has power and influence. 
They recognize that virtually every decision 
has trade-offs, and they deliberately weigh 
the advantages and disadvantages of alterna- 
tive courses of action. They use available 
resources efficiently and effectively, and are 
opportunistic in leveraging additional 
resources. Strategic leaders create and take 
advantage of opportunities for increasing the 
depth and coherence of reform, and are 
explicit about what aspects of reform are crit- 
ical to long-term success. They are able to 
keep in mind the long-range vision and "big 
picture" of system reform, even as they make 
the myriad of day-to-day decisions involved 
in their work. Finally, strategic leaders use 
the external funding resources as a catalyst to 
set in motion other forces that can produce 
change in the system as a whole. 

The HRl study was grounded in an 
understanding that the SSls were intended to 
create substantial and enduring reform in 
education policy, administration and man- 
agement, support services, and the teaching 
and learning of mathematics and science. The 
following lessons learned from these endeav- 
ors elaborate on some of the key findings of 
our study. The lessons reflect the successes, 
struggles, and experiences of those involved 
in the SSls. We believe these lessons can pro- 
vide some guidance to others who lead, over- 
see, provide assistance to, evaluate, and 
make decisions about current and future 
large-scale education reform efforts. 
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Lessons Learned 

• Strategic reform leaders have a vision of a 
reformed system and how to achieve it. 

A vision for effective teaching and learn- 
ing is not enough to guide large-scale reform. 
Rather, strategic leaders possess a larger view 
of reform and a road map for getting there. 
As one PI said, "To have a vision is one thing. 
To be able to visualize how you are going to 
enact it would be something different." Typi- 
cally, SSls drew their vision of high-quality 
mathematics /science teaching and learning 
from national standards documents. In some 
cases, that was the extent of the vision: As 
long as a proposed intervention or activity 
was consistent with national standards, or 
the state's interpretation of those standards, it 
was deemed appropriate for the SSL As a 
result, SSls often found themselves scram- 
bling to implement a host of interventions, 
but without a set of criteria for ensuring that 
these activities were part of a coherent plan 
that produced system-wide change. 

In contrast, strategic leaders articulated a 
vision that went beyond national standards 
documents. Some recognized the importance 
of this viewpoint from the outset. Others 
came to realize over time that simply spon- 
soring a myriad of standards-aligned activi- 
ties was insufficient for producing the kind of 
long-term, system-wide impact desired. 
Their vision included a view of the kind of 
system that would support high-quality 
mathematics and science instruction, and a 
plan to reform the system accordingly. One 
SSI leader summed up the frustration of con- 
vening existing projects to involve them in 
the initiative without first developing this 
level of understanding: 

It wasn’t a unified effort of how we could 
come together to solve a problem, [but rather] 
“It’s our little thing and that’s all we have to 
do and we’re not really involved in the bigger 
problem." . . . Because of that initial mistake, 
it was a battle from then on in. . . . The way 
that the proposal was written is that we got 
groups together on numerous occasions and 
we’d say, “All right, write us a little propos- 
al of what you ’d like to do" without any 
guidance or general sense of direction. So we 
just kept trying to fix that all the way 
through. 



For strategic leaders, the vision for how 
the SSI could lead to reform represented a 
plausible process. The vision was context- 
sensitive, since it represented a design for 
moving a particular education system from 
its present condition, with all of its contextu- 
ally dependent policies and practices, toward 
an envisioned future condition. Virtually all 
of the SSls espoused a vision of excellent and 
equitable teaching and learning and had a 
number of ideas for reform activities. In con- 
trast, the more strategic SSls were able to 
explicitly link vision and strategies, respond 
to context without altering long-range goals 
for change, and specify how the particular 
activities would lead to system changes that 
supported excellent and equitable teaching 
and learning. 

• Strategic leaders cultivate broad under- 
standing and support for the reform vision at 
the highest levels. 

Nearly every SSI established a steering 
committee or major board that included 
highly placed stakeholders in the state. The 
SSls frequently involved the chief state 
school officer, college and university presi- 
dents, legislators, and representatives of the 
business community in their efforts. In inter- 
views, several Pis talked about regular meet- 
ings with the governor's education advisor, 
or with the staff of key legislators, to keep 
them apprised of SSI activities and to keep 
the SSI informed of priorities and/ or shifts in 
policy direction. 

In terms of political strategizing, involv- 
ing these key power brokers was imperative. 
In some cases, SSls involved state-level stake- 
holders by strategically linking the SSI to an 
existing, influential group. For example, the 
South Carolina SSI crafted its design after a 
report issued from the Mathematics and Sci- 
ence Advisory Board (MSAB), a highly visi- 
ble and broadly based group appointed by 
the governor and state superintendent. The 
SSI was sponsored by the MSAB as a new 
entity within the Department of Education, 
virtually assuring the support of key state 
leaders in government, higher education, 
business, and K-12 education. The backing of 
the MSAB and the state superintendent per- 
suaded other important constituencies to 
support the SSI as well. 
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Other SSIs tied their projects to existing 
reform movements initiated by key players in 
the education system as a way to garner the 
early support of influential stakeholders. 
Connecticut, Delaware, Michigan, Puerto 
Rico, Texas, and Vermont all took this 
approach. Even when the other initiatives 
were fledgling, linking the SSI to these efforts 
attracted the support of those driving the 
reforms. Governors and state superinten- 
dents again were frequently key players; 
business and industry leaders often voiced 
solid support for these reform movements as 
well. 

The SSIs were uniquely positioned to play 
a convening role in unifying some of the 
divergent reform efforts, and strategic lead- 
ers took advantage of the possibilities this 
role offered. Including diverse and powerful 
state stakeholders yielded positive results — 
not only for the SSIs, but also in terms of the 
interest and involvement of these constituen- 
cies in mathematics and science reform in 
general. Leaders in Maine and Michigan 
recalled that 

[We] were bringing groups of people togeth- 
er that had never been together before, and 
talking about what was important about 
math and science education in Maine for all 
students, and where we were and what could 
be done, and there was just such enormous 
leverage in that. 

The [Michigan ] SSI provided a way to con- 
vene people, which hadn't been happening. 
There were all these pieces out there, but no 
entity to convene the players. So they got 
people together and got them talking in the 
same ways and the vision emerged from that 
work. 

Some SSIs designed specific interventions 
to respond to the interests of key stakehold- 
ers as a way to engage them more fully in the 
reform. Michigan, for example, expanded a 
component targeting pre-service education 
because of the input of influential partici- 
pants in the SSL Maine appointed key busi- 
ness leaders to head its community outreach 
and systemic plarming components. Con- 
necticut crafted its plan specifically to 
respond to the interests of key stakeholders, 
expanding efforts in pre-service education as 
Michigan did, and seeking specific ways to 



engage stakeholders as Maine did. In Con- 
necticut, these changes were prompted by the 
state's early commitment to use the SSI as 
part of a 12-year plan for reform in mathe- 
matics and science education. The PI reflect- 
ed that 

Some people said, “You know, we're never 
going to get people to stick around for 12 
years." Well, no, [unless] they have a piece 
that they can play, not necessarily money 
that they get, and they can begin to see that 
we as a state are making movements toward 
our combined goals. People have all kinds of 
longevity. 

To ensure that equity goals retained their 
prominence in the SSIs, some leaders strate- 
gically chose to include representatives of 
traditionally underrepresented groups in the 
highest levels of SSI management and opera- 
tions. The Delaware SSI, for example, made a 
point to involve not only scientists, mathe- 
maticians, science and mathematics educa- 
tors, and administrators from Delaware State 
University, but also representatives from 
community organizations serving historical- 
ly underrepresented children, such as the 
Boys Club of Delaware and Big Brothers /Big 
Sisters. Similarly, the Georgia SSI engaged an 
ethnically diverse leadership team, and 
sought to establish reform centers around the 
state housed in universities serving specific 
regions, including some serving primarily 
minority populations. 

• Strategic leaders cultivate the commitment 
of school and district leadership to the reform 
vision and activities. 

Concurrent trends in most states toward 
centralized policy guidance on the one hand, 
and decentralized decision making to 
respond to policies on the other hand, had 
implications for cultivating broad under- 
standing and support for the SSI vision. It 
meant that SSIs needed a presence at both the 
state and district levels. Much of the work of 
the SSIs, however, focused on engaging the 
more powerful, state-level constituents who 
were well positioned to influence policies 
and practices around mathematics and sci- 
ence education. In fact, SSIs often neglected 
administrators at the school and district lev- 
els — stakeholders whom many SSI leaders 
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found to be more crucial than they had origi- 
nally anticipated. For example, in Nebraska 
one of the SSI Pis recalled that the regional 
partnerships and scaling-up efforts went 
"well beyond the boundaries of the grant" in 
districts where the superintendent and prin- 
cipals demonstrated a solid commitment to 
the reform. One leader recalled: 

Very early there was a lot of focus on build- 
ing the capacity of teachers, not only to teach 
well but also to lead the charge, if you will. 
One of the mistakes in those early days is we 
didn't recognize how important the role of 
formal leadership was, and in many cases we 
created some high-powered teachers ready to 
lead in their schools, and their principals or 
superintendents weren't on board at all. I 
think we learned from that, but it was an 
issue early on. 

The experiences of the SSls suggest sever- 
al opportunities to cultivate the will and 
commitment of local leaders to support 
reform. For example, local committees often 
make decisions on adopting instructional 
materials that play a key role in defining con- 
tent and pedagogy in the classroom. Ensur- 
ing an SSI presence on these committees 
increased the likelihood of establishing crite- 
ria and processes for selecting high-quality 
materials aligned with the reform vision. 
Similarly, principals and other local leaders 
evaluate teachers and make decisions about 
the nature and extent of teacher professional 
development. Several SSls claimed successes 
in changing the views of effective profession- 
al development among local decision makers 
and saw these changes as representative of a 
significant and lasting accomplishment. An 
SSI leader in Arkansas highlighted this 
accomplishment: 

I think what the SSI did was it showed 
administrators that you didn't fix the prob- 
lem with a one-day math workshop. ... I 
think we educated them here in the SSI [that] 
that's what the research said about changing 
student improvement, that it's a multi-year 
process, that it has various components. I 
think that the thing that the SSI did that no 
report could ever reflect is that it educated 
administrators about what meaningful staff 
development would be. 



Assuming that individuals and organiza- 
tions would operate on the basis of rational 
choices (Heywood, 1999) that served their 
own "enlightened self-interest," strategic SSI 
leaders sought mutually beneficial solutions 
with local constituencies. For example, a 
leader of the Vermont SSI described how the 
initiative repositioned itself to help school 
leaders fulfill policy requirements that were 
called for by the state. By responding to local 
needs through its own work, the SSI was able 
to heighten awareness of, and commitment 
to, mathematics and science education 
reform: 

[The new legislative act] set up expectations 
for schools that were different from what had 
been. One was that schools would start using 
data in more serious ways, that schools 
would examine their student performance 
data in the context of other data and create an 
annual action plan to focus professional 
development, resource allocation, energy, etc. 
That's made a huge difference. What we've 
had to do as an organization is to become 
responsive to the action plans. If schools have 
decided where they're going to put their 
emphasis and spend their time in profession- 
al development, we need to be part of helping 
them get what they need. So we spend a lot 
more time helping people do data analysis, 
translating the data into action plans, and 
then implementing those plans. That has 
fundamentally changed the way we work. 

The changing nature of the responsibili- 
ties of school and district administrators, 
especially in the areas of data-informed deci- 
sion making and accountability, provided 
opportunities for SSls to influence local prac- 
tices. As in Vermont, administrators through- 
out the United States are increasingly being 
required to collect, analyze, report, and use 
data in their decisions and actions. Many SSls 
found themselves in strong positions to work 
with administrators to develop systems and 
capacities for these new responsibilities, and 
consequently to introduce a systemic view of 
management in many localities. 
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• Strategic reform leaders use interventions 
to translate vision into practice. 

Just as "actions speak louder than 
words," the interventions of a reform can 
communicate what the vision is all about. A 
mission statement that seeks to communicate 
the overall vision and direction of reform 
may do so with a limited audience. In con- 
trast, interventions are there for the broader 
public eye to take in. They can put a face to 
reform. Operationalizing the reform vision 
through interventions is a way to demon- 
strate commitment to ideals and to show how 
that commitment translates into real efforts 
and activities to change the system. 

Most SSls were able to justify interven- 
tions based on need in virtually any place in 
the system. Strategic reform leaders, howev- 
er, focused their interventions on particularly 
needy parts of the system and/or places 
where there were opportunities for consider- 
able leverage. A leader in the New Jersey SSI 
explained the reasoning behind that SSI's 
choice of teacher professional development 
at the elementary and middle grades as areas 
of intensive initial focus, coupled with com- 
ponents targeting a broader scope of the sys- 
tem: 

We felt the professional development void 
was greatest in the K-8 level. We only had a 
K-8 certificate, not even a middle school cer- 
tificate. This was a pressing need. We also 
felt we would have some difficulty making a 
real impact at the high school level if we did- 
n’t have impact first at the K-8 level. We also 
felt that elementary and middle schools 
would be more receptive because they didn 't 
have entrenched departmentalization. 

Many SSls stressed the critical role of 
teachers in reform through a large invest- 
ment in professional development, attending 
to teachers' knowledge and skills as neces- 
sary for substantive changes in teaching and 
learning. This approach built credibility 
among key constituents but sometimes 
revealed a more limited vision as discussed 
above — one focused solely on improved 
teaching and learning in mathematics and 
science. The more strategic leaders also 
planned for other kinds of interventions and 
activities directed toward broader con- 
stituencies — administrators, business, uni- 



versities, and communities — and used these 
interventions to demonstrate commitment to 
system-wide change. 

• Strategic reform leaders recognize the ben- 
efits of starting small, refining activities as 
needed, and providing evidence that inter- 
ventions lead to the desired outcomes. 

SSI leaders learned the value of starting 
out with a manageable scope and scale. In 
cases where the interventions required more 
capacity or resources than the SSI could 
apply on a large scale, or in cases where pro- 
fessional or public support was mixed, start- 
ing small helped ensure that reform activities 
could be implemented as planned, with ade- 
quate staff and resources to work out the 
kinks and gather evidence of impact. Some- 
times this meant a less than linear path for 
the SSls. For example, a leader in the Texas 
SSI explained why the initiative engaged in 
some direct work with districts, even though 
the reform plan ultimately called for work 
through intermediary organizations, such as 
regional education service providers: 

[We believed that] if you teach the standards 
and you teach them well, as they were 
designed, you will see high levels of perfor- 
mance on the test. That influenced us to work 
in some districts to make sure it happens, 
because just working through intermediaries, 
we’ve got (quality control issues. And so 
we’ve tried to work directly in some districts 
. . . to learn what it takes to make that hap- 
pen. 

In particular, strategic SSI leaders recog- 
nized the benefits of monitoring and evaluat- 
ing interventions. Those who established 
data systems could track quality and impact, 
and use the information to improve interven- 
tions over the course of the SSI. Strategic 
leaders also used this evidence to demon- 
strate that the interventions were producing 
the desired results, and to secure support for 
expanding their efforts. For example, the 
Ohio SSI prepared a "Pocket Panorama" with 
bar graphs showing a narrowing of perfor- 
mance gaps between White and minority stu- 
dents, based on the SSI's administration of a 
test comprised of NAEP released items and 
linked to the SSI interventions. The results 
showing an impact on student understand- 
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ing were distributed to key individuals in the 
state legislature, and greatly increased poli- 
cymakers' interest in the SSL Said a PI in 
Ohio: 

We began collecting student data . . . and 
began our public relations campaign with 
those “Pocket Panoramas." They went out to 
every legislator and every superintendent. 
We had to build a constituency. . . . You 
know, to keep funding you have to show them 
kids are learning. 

Similarly, the Puerto Rico SSI spent con- 
siderable resources developing, administer- 
ing, scoring, and analyzing pre-post student 
assessments as the initiative scaled up its 
efforts. According to the PI, "it was money 
well spent" for demonstrating impact and 
gaining support for reform. The data showed 
that the interventions not only produced pos- 
itive results in the small number of "Phase 1" 
schools where the initiative's staff provided 
direct services and assistance, but also in the 
much larger number of "Phase 11" schools 
where the faculty and administration from 
Phase 1 schools provided the majority of ser- 
vices and assistance. 

• Strategic reform leaders develop "system" 
capacity to scale up reform with quality. 

SSI leaders learned that many kinds of 
capacity are needed to implement and sus- 
tain systemic reform: human capacity to 
deliver the interventions; organizational 
capacity to house and support reform; and 
leadership capacity to direct, manage, and 
monitor reform. Such capacities were sparse 
in most states when the SSls began. Even in 
small states, the SSls initially lacked the 
capacity needed to reach all districts, schools, 
and classrooms. 

Our study reinforced the view that a 
reform "initiative" can be a catalyst that sets 
a change in motion, rather than a force that 
produces all of the desired changes. Ulti- 
mately, interventions for large-scale reform 
have to expand well beyond the initiative's 
early participants. In addition, reform efforts 
require a system for quality control during 
scale-up to ensure that interventions have the 
desired impact. SSI leaders learned that scal- 
ing up too quickly made it difficult to main- 
tain quality. On the other hand, scaling up 



too slowly made it impossible to have a 
broad impact. Strategic SSI leaders recog- 
nized the importance of developing feasible 
plans for scaling up within a reasonable time 
frame, and devised strategies for dealing 
with the inherent tension between the need 
for scale-up and the need for quality control. 

The challenges in tending to this tension 
were abundantly evident in many SSls. For 
example, quite a few of the SSls worked to 
develop teacher leaders as a means of scaling 
up their initiatives. SSls engaged these indi- 
viduals in fairly intensive professional devel- 
opment and then arranged for them to pro- 
vide workshops to other teachers, in some 
cases in teams that also included scientists or 
mathematicians as content experts. SSI lead- 
ers and evaluators who observed workshops 
and institutes provided by teacher leaders 
typically reported considerable variation in 
quality; they concluded that the level of pro- 
fessional development was insufficient for 
enabling many of the teacher leaders to pro- 
vide high-quality professional development 
to their peers or to otherwise serve as leaders 
in their districts and schools. 

The more strategic SSI leaders dealt with 
this challenge directly. For example, a leader 
in the Vermont SSI described how the project 
altered the design to provide additional sup- 
port to teacher leaders: 

The original model was based on a faith in 
the ability of professional development to 
enable teachers to go back and be leaders in 
their schools. What [the SSI] found over the 
first three or four years was that that was 
important, but it wasn’t sufficient. The 
thought was that if we can bring teachers in 
to work in a leadership capacity in [the SSI] 
for a couple of years, get intensive experience 
in content, in pedagogy, in leadership, then 
go back to their schools, they’ll have the lead- 
ership skills to move things forward. 

A similar strategy was used in the 
Delaware SSI, where the initiative created a 
cadre of teachers and community 
members — mostly other school-based pro- 
fessionals — who would be available as "cul- 
ture-change agents" to provide technical 
assistance to schools. Of course, the strategy 
of developing teacher leaders who could then 
provide direct services to a large proportion 
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of teachers in the state was far less feasible in 
larger states, some of which had as many 
teachers as Vermont or Delaware had stu- 
dents. Several SSls tapped into existing infra- 
structure, such as regional service centers, to 
scale up the reforms, but faced challenges in 
ensuring a common vision and consistently 
high-quality interventions. Those that built 
their own regional infrastructure for scaling 
up services faced similar challenges. As one 
leader commented, "Every [regional group] 
needed someone with vision, and only about 
half of them had it." 

A different set of challenges arose for SSls 
that adopted "model" sites as a scaling-up 
strategy. Several of the initiatives invested 
considerable resources in developing capaci- 
ty for high-quality reform in a few schools or 
districts with the expectation that the quality 
of change in those sites would inspire other 
schools and districts to change, and would 
prompt the state to support similar capacity- 
building efforts on a larger scale. However, 
few of these SSls were able to convince fun- 
ders or legislatures to make such an invest- 
ment, and efforts to expand the models could 
not be supported through the initiative's 
resources alone. 

SSI leaders pointed to the benefits of 
building a feasible scaling-up strategy into 
the design from the outset. For example, the 
Puerto Rico SSI made its strategy for scaling 
up explicit from the beginning, laying out a 
plan to reach a sizable proportion of the 
schools in the Commonwealth. The design 
included strategies for building capacity and 
credibility through piloting, evaluating, 
revising, and expanding activities. Leaders 
were deliberate in their choice of reform 
strategies — starting small, using the school 
as the unit of change, and using regional cen- 
ters as "test beds." Each step would lay the 
groundwork for the next, with reform leaders 
moving carefully and deliberately, building 
the capacity of core team members, major 
partners, and school staff along the way to 
ensure readiness for change. More typically, 
however, SSls had some notion initially of 
how they might scale up, and devised other 
strategies as the initiatives unfolded. 

From the HRl study, no single strategy for 
scaling up emerged as optimal across con- 
texts. Rather, it was the process and consider- 



ations behind developing a strategy for scal- 
ing up that pointed to a level of effectiveness. 
SSI leaders who took into account a reason- 
able rate of growth, quality control, and a 
sustainable infrastructure characterized the 
more successful approaches. Lacking any one 
of these considerations, SSls struggled to 
scale up reforms. 

• Strategic leaders facilitate the development 
of formal policies that provide guidance and 
incentives for the reform vision. 

One of the hallmarks of systemic reform is 
the focus on education policy as well as ser- 
vice delivery. Education policies provide 
guidance and incentives for districts, schools, 
teachers, and others. Policies also require 
these players to pay attention to specific 
aspects of teaching and learning, and to take 
advantage of particular services. Although 
SSI leaders had generally not been active 
players in shaping education policy previ- 
ously, many took on this aspect of the work 
as a necessary part of their role. 

SSI leaders positioned their projects to 
engage in policy work in various ways. For 
example, while some established close orga- 
nizational ties to agencies that developed or 
influenced policies, SSls that operated out- 
side of formal education agencies often 
included individuals with policymaking 
influence as leaders in the initiative. For 
example, the Puerto Rico SSI was established 
as a partnership among three vital agencies 
in the education system of the Common- 
wealth — the Puerto Rico Department of 
Education, the governor's General Council 
on Education, and the Resource Center for 
Science and Engineering at the University of 
Puerto Rico. In addition, the PI had an estab- 
lished reputation for high-quality work in 
education and a position on the Council of 
Advisors to the governor. 

The South Carolina SSI was similarly 
positioned to engage in policy work relevant 
to mathematics and science education. Close- 
ly tied to the South Carolina Department of 
Education through its leadership and organi- 
zational home, the SSI enjoyed strong sup- 
port from the state superintendent. Given 
this position, it was a "natural" for SSI lead- 
ers to work on policies in mathematics and 
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science education. When the focal policies in 
the state shifted from curriculum alignment 
to school improvement plarming, the SSI's 
organizational position and high-level lead- 
ership allowed it to retool its policy focus, 
while keeping mathematics and science cen- 
tral to policy decisions. 

In Michigan, the SSI was inaugurated 
within the state Department of Education, 
and included key leaders in mathematics and 
science education in the initiative's leader- 
ship. Because it began in a system that was 
perceived already to have a fairly supportive 
policy environment for systemic improve- 
ment, one observer said, "[The Michigan SSI] 
looked to see where there were gaps. For 
example, they chose not to focus on issues 
around standards because [that] was already 
in the works."Rather, the SSI defined its role 
in the policy arena as that of a critical friend, 
reviewing existing policies and their implica- 
tions for districts and schools, and making 
recommendations for policies with greater 
"power and coherence" in mathematics and 
science, especially for underserved students 
in the state's schools and districts. Michigan 
SSI leaders in positions of influence would 
use the reviews and recommendations to 
leverage changes in the policy arena. 

Making a valued contribution in one pol- 
icy area sometimes opened doors for SSls to 
play an expanded role in other policy areas. 
In some cases, SSls took the lead in develop- 
ing position papers, frameworks, or reviews 
that became important resources for subse- 
quent policy developments in their states. 
These products signaled the intent of the SSI 
to be a player in the policy arena and demon- 
strated the capability of the SSI to do so. The 
remarks of one leader echoed the story of 
other SSls: 

[The SSI] made a conscious decision to be an 
active partner in the development of the 
statewide frameworks. That was very impor- 
tant, because they became the foundation for 
other work in the state. And through that 
process [the SSI] was able to build some real 
credibility for its capacity in math and sci- 
ence. 

By monitoring education policies, SSls 
could also develop an "inside track" on what 
was being offered to, and required of, dis- 
tricts and schools. Some SSI leaders took par- 



ticular advantage of such information to 
reposition their initiatives to help local 
administrators meet policy requirements. In 
the process, strategic SSI leaders were able to 
integrate the expertise, services, and systemic 
perspective of the initiative into the work of 
districts and schools. 

• Strategic leaders avoid becoming a light- 
ning rod for controversy. 

The intent of the SSls to reform the educa- 
tion system was bound to raise some skepti- 
cism, criticism, and perhaps even opposition. 
However, reform rests on the belief that the 
capacity to improve and to address persistent 
problems resides, or can be developed with- 
in, the existing system (Heywood, 1999). Fos- 
tering reform required the SSls to engage 
with their skeptics, critics, and opponents in 
productive ways to build a common vision. 
As one SSI PI noted, "Anyone who doesn't 
understand that reform is a political game 
doesn't know anything about reform." 

Some of the more strategic SSI leaders 
were able to establish neutral political turf to 
bring constituencies together. Maintaining a 
connection to, but a reasonable distance 
from, existing agencies allowed these initia- 
tives to become conveners of a broad array of 
stakeholders and to build a reputation for 
facilitating collaborative solutions. One strat- 
egy for establishing neutral political territory 
was to house the SSI in an independent, non- 
profit organization, as one leader in Maine 
described: 

The importance of the SSI in Maine choosing 
to go to a nonprofit structure was very valu- 
able, because it allowed that group to be flex- 
ible, to shift and change, and not necessarily 
be politically aligned anywhere. It is a high- 
risk strategy in the sense that we could be 
disregarded, but you aren't pigeonholed any- 
where. So you can take the high road without 
having to get dragged down by other values 
... in the particular institution. 

Working with schools and classrooms 
required the SSls to nurture relationships 
with those who translated state policies into 
local practices. Parents and other local stake- 
holders needed to come to understand, and 
more importantly to favor, the kind of teach- 
ing and learning that SSls were advocating. 
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In particular, an initiative devoted to equity 
had to find ways to bring together the tradi- 
tionally powerful and the traditionally 
underrepresented voices in mathematics and 
science education. 

The Texas SSI used an innovative strategy 
to engage leadership around the state, to 
maintain a high statewide profile, and to 
make the work of the SSI as politically inclu- 
sive and neutral as possible. Involving stake- 
holders in substantive reform work was one 
of its primary missions. The SSI committed 
itself to having diverse viewpoints to repre- 
sent particular issues in mathematics and sci- 
ence education, with the SSI playing a back- 
ground role to facilitate buy-in to the reform. 
Said one SSI leader from Texas: 

We knew that if we could become the conven- 
er, we wouldn’t have to be the leader, if you 
will, because we could facilitate and lead 
through facilitation and consensus building, 
because Texas is a very independent state and 
it 's kind of hard to be the leader. There are too 
many leaders, and they're not willing to give 
up their power and authority, so that's the 
strategy we’ve used, and it’s worked pretty 
well. It’s worked better in math than in sci- 
ence; science is still a bit fragmented, but 
we’re beginning to build a coalition. So that 
was part of what we were thinking about way 
back there. We wanted to be the convener, the 
consensus builder, around the state, and I 
think people would say that we are at this 
point. 

Producing deep and lasting change in 
mathematics and science education systems 
required that SSIs make critical trade-offs in 
the ways they positioned themselves politi- 
cally. Close ties to powerful agencies or indi- 
viduals offered opportunities to influence 
policymaking, but could also pigeonhole the 
SSI and hamper its ability to involve a broad- 
er array of stakeholders. Further, reform 
leaders could not afford to ignore policies or 
influential constituencies, even if they were 
not entirely aligned with the SSI's goals. 
Finally, in seeking to transform mathematics 
and science teaching and learning, SSI lead- 
ers had to avoid getting so far "in front of the 
curve" of reform that districts, schools, and 
teachers could not keep up with changes. 



• Strategic leaders develop the capabilities of 
the next generation of reform leaders. 

Limited time and resources require that 
large-scale reform efforts "grow" new and 
expanded leadership — leaders who can 
carry the reform into the future. One Puerto 
Rico SSI staff member who developed into a 
strong leader through the SSI experience 
commented: 

NSF’s (question early on made perfect sense. 
“What’s going to happen to the [SSI] if the 
plane that [the PI] is on goes down? Will the 
[SSI] go down too? Or are there other people 
who can take over?" At the beginning of the 
project, we said, “Well . . . we’re working on 
that." And now we can say categorically yes. 
We have more experience, more exposure, 
and more mileage now. We have grown into 
our roles. And the way we play our roles and 
our strengths, we complement each other. 
[The PI] allowed leadership to develop. 
There’s good vision and good teamwork. 

It is not sufficient, however, for an initia- 
tive to develop new leaders without also con- 
sidering how to position those leaders to 
exert influence and authority. A leader of the 
SSI in Nebraska remarked on the conse- 
quences of failing to undertake this key task: 

The legacy is really held within the people 
who had this experience. The legacy is not 
really held within the basic organizational 
changes. I’m afraid that with SSIs, that was 
the original intent — that organizations and 
the way of doing business would change so 
dramatically that the infrastructure would be 
there as people came and went. In reality, I 
think the legacy is held within the people 
rather than in a major lasting infrastructure. 

It has changed the way people do business 
and made them more connected, but when 
they leave, the legacy goes with them. 

To deal with this dilemma, some SSIs 
arranged for positions within the existing 
education system where new leaders could 
develop their full potential and continue to 
support reform. For example, the Arkansas 
SSI managed to keep some of the SSI activi- 
ties in place, despite the fact that the SSI no 
longer exists as an organization. In this case, 
toward the end of the funded period, the pro- 
ject director presented a plan to the Depart- 
ment of Education to support mathematics 
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specialists throughout the state, including 15 
in the existing regional service centers and 11 
in the universities that had participated in the 
SSL The SSI trained the mathematics special- 
ists, and the former project director continues 
to coordinate their activities by meeting with 
the directors of these centers: 

Once every six weeks or so I call a directors' 

meeting And we come together and share 

what we've been doing in our different 
regions. We have two or three state projects a 
year that we work on together. They all have 
assignments. I go visit them. I guess it's psy- 
chological; I still have the role of the director 
of SSL People still say that. I just carry that 
on. That's kind of a perception that I just 
haven 't allowed to let go on their part. 

A number of other SSls were able to estab- 
lish organizational entities that continue to 
provide a base for reform leadership, such as 
nonprofit agencies and centers within uni- 
versities. These organizations have been sus- 
tained through the commitment of state 
funds, fee-for-service arrangements with dis- 
tricts, and public and private grants, and they 
provide a "home" for those leaders cultivat- 
ed under the SSI to continue their mission. 

Conclusion 

The study of strategic leadership in the 
SSI program provided an opportunity to 
examine cases of federally supported, large- 
scale reform in mathematics and science edu- 
cation, even beyond the funding period. The 
notion of techrrical and political challenges 
was a useful analytic distinction for the 
study, and we suggest it can be a helpful lens 
for leaders of large-scale reform as they 
examine their plans and reflect on their work. 

Many of the lessons that emerged around 
strategic leadership focused on political chal- 
lenges rather than technical ones. Most SSI 
leaders were already adept at many technical 
aspects of reform, and they demonstrated 
proficiency in their proposals. By contrast, 
the political aspects of reform came as a sur- 
prise to many SSI leaders, and were a key 
area of learning. A nearly universal lesson 
was that attending to both technical and 
political aspects of large-scale reform is criti- 
cal. Reform leaders ignore either at their 
peril. 



The technical and political aspects of con- 
ducting large-scale reform are not easily sep- 
arated, nor should they be. Nearly all deci- 
sions will have both technical and political 
considerations. Highly strategic approaches 
seek a balance — where neither technical 
concerns for quality and impact, nor political 
concerns for guidance and incentives, are sac- 
rificed. Rather, these concerns become mutu- 
ally reinforcing and appropriate to the broad- 
er context of the system. V^en compromises 
occur, often due to contextual factors, they 
reflect deep consideration of the trade-offs 
and do not violate the technical and political 
considerations needed most for advancing 
the reform. 

Highly strategic approaches to large-scale 
reform are also characterized by leverage and 
synergy — by the leverage of policies and 
programs in the education context, and by a 
synergy that extends over time so that con- 
current efforts reinforce one another and each 
phase of activity sets the stage for the next. 
For example, a strategic initiative might 
begin by conducting in-depth professional 
development and curriculum implementa- 
tion, with strong quality control, in a small 
number of schools or districts. Such a plan 
affords the opportunity to reinforce the 
vision of teaching and learning, to test and 
refine interventions, to provide "existence 
proofs" that the interventions produce the 
promised results, to build capacity for scaling 
up the interventions, and to use results to 
influence policy decisions. 

An alternative approach might begin with 
a review of state and district policies and pro- 
grams to understand where there is align- 
ment with the reform vision and where align- 
ment is lacking. Reform leaders could use the 
review to identify "aligned" and "unaligned" 
sites, matched demographically in order to 
compare teaching and learning outcomes. 
Such a study has the potential to provide evi- 
dence that aligned policies and programs are 
related to the desired results, and the study 
could be presented to various stakeholders to 
cultivate support for reform. Beyond the evi- 
dence phase, aligned sites might, with the 
guidance of the initiative's leaders, provide 
assistance to other less aligned sites. The 
strategy might also include a process for 
developing local leadership and capacity to 
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conduct policy and program reviews to bring 
more coherence to the local system. 

Though overly simplified here, these 
designs make explicit the assumptions, pro- 
gression, and interface of reform strategies, 
and take into consideration both short- and 
long-term goals and objectives. Plans for both 
scaling up and going to scale are included at 
the earliest stages. Creating such a design is 
no guarantee of success. However, it pro- 
vides a necessary frame for guiding imple- 
mentation, for dealing with unanticipated 
challenges and opportunities, and for inform- 
ing decisions in the always unpredictable 
and often volatile education system. 
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Recent CPRE Publications: 

Special Issue: CPRE Researchers Contribute Knowledge of Assessing Teacher, Classroom, and 
School Effects In the Peabody Journal of Education 

CPRE researchers at the University of Wisconsin-Madison served as guest editors in the recently released issue of the Peabody Journal of Edu- 
cation. The researchers contributed six articles focusing generally on new directions in assessing and measuring teacher, classroom, and school 
effects on improvements in student academic achievement, and more specifically analyzing the criterion vahdity and surrounding human 
resources strategies of new efforts to implement performance-based teacher evaluations — the results of which in some cases are linked to new 
knowledge-and skills-based teacher salary schedules. A brief description of each article follows. Please e-mail cpre@gse.upenn.edu to obtain 
copies. 

Introduction to the Special Issue: Assessing Teacher, Classroom, and School Effects (Allan Odden) 

This introductory article explains the purpose and sequence of the six articles, providing background on the inter-related focus and overall con- 
clusions drawn from the research. 

Assessing Teacher, Classroom, and School Effects, Including Fiscal Effects (Allan Odden, Geoffrey Borman, and Mark 
Fermanich) 

This article provides a general conceptual framework for the three articles to follow. The focus of fhis article surrounds the authors' position that 
too much previous research has tended to assess the effects of student, classroom, and school variables in isolation from other variables and has 
often used statistical techniques that ignored the nested nature of the three classes of factors. The authors identify an alternative educationally 
oriented framework to assess the effects of various student, classroom/ teacher, and school variables on student learning. 

The following articles use the developed general framework to study the criterion validity of new performance-based teacher eval- 
uation systems, which provide one measure of teacher quality. All three places described below use some version of the teaching 
standards and evaluation rubrics developed by Charlotte Danielson. 

The Relationship Between Teacher Performance Evaluation Scores and Student Achievement: Evidence From 
Cincinnati (Anthony Milanowski) 

This article studies the system in Cincirmati, Ohio, the results of which were initially to be linked to a performance pay structure. Milanowski 
presents the results of an analysis of fhe relationship between teacher evaluation scores and student achievement on district and state tests in 
reading, mathematics, and science. 

Examining the Relationship Between Teacher Evaluation and Student Assessment Results in Washoe 
County (Steven Kimball, Brad White, Anthony Milanowski, and Geoffrey Borman) 

Kimball et al. study the system in Washoe County (Reno, Nevada), which is not linked to pay, and describe the findings from an analysis of fhe 
relationship between the scores on a standards-based evaluation system and student achievement measures. 

Vaughn Elementary's Innovative Teacher Evaluation System: Are Teacher Evaluation Scores Related to Growth in 
Student Achievement? (H. Alix Gallagher) 

This article examines the vahdity of a performance-based, subject-specific teacher evaluation system by analyzing the relationship between 
teacher evaluation scores and student achievement. 

Alignment of Human Resource Practices and Teacher Performance Competency (Herbert Heneman III and Anthony 
Milanowski) 

The authors of this article argue that human resource (HR) management practices are important components of strategies for improving student 
achievement in an accountability environment and that these HR systems, within which ambitious evaluation and compensation systems are 
managed, need to be strengthened. The authors conclude with recommendations on developing more effective HR systems. 

Lessons Learned About Standards-Based Teacher Evaluation Systems (AUan Odden) 

This concluding article outlines a set of "lessons learned" from the above articles, focusing largely on the lessons learned about designing, imple- 
menting, and using performance-based evaluation systems that could be linked to teacher play increases. 
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